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Abstract 

We discuss design techniques for catadioptric sensors that realize given projections. In general, 
these problems do not have solutions, but approximate solutions may often be found that are visually 
acceptable. There are several methods to approach this problem, but here we focus on what we call the 
"vector field approach". An application is given where a true panoramic mirror is derived, i.e. a mirror 
that yields a cylindrical projection to the viewer without any digital un warping. 

1 Introduction 

A fundamental problem of imaging is the creation of wide-angle and panoramic images. Wide-angle images 
that do not have radial distortion require complex assemblies of lenses, because as the field of view increases, 
abberations are introduced, which in require the addition of more lenses for correction. Fish-eye lenses offer 
wide-field of views but at the cost of considerable radial distortion. This distortion may be removed in 
software, but the resulting images suffer from poor resolution in some regions. 

Panoramic images present a related challenge. A straightforward, yet tedious approach is the stitching 
together of images. In the 19th century numerous cameras were created for taking panoramic still shots 
through ingenious rotating mechanisms (e.g. rotating slit cameras), but were inherently awkward. Adapting 
such devices for digital imaging (especially video applications) is clearly problematic. 

The introduction of a curved mirror into a conventional imaging system is an elegant solution to these 
problems. General design questions are: 

A. How does one design a catadioptric sensor to image a prescribed region of space ? 

B. How does one design a catadioptric sensor to maximize image quality ? 

C. How does one design an optimal catadioptric sensor for a given machine vision application, such as robot 
navigation, tracking or segmentation ? 

Commercial interest in panoramic sensors based on these ideas has risen sharply in the last five years, 
and a large number of companies have emerged to to meet the demand. Common commercial applications 
are surveillance and "virtual tours" on the web. At last count, at least 19 different companies were found 
offering panoramic sensors for sale, the majority of which were based on catadioptrics (see |4]). 



2 Related Work 



2.1 Rotationally Symmetric Mirrors 

Almost all research performed with mirrors refers to rotationally symmetric mirrors, since these mirrors 
are the simplest to make and to mathematically model. For every curve in the plane, one may create a mirror 
by rotating the curve about a chosen line, which will then play the role of the optical axis. Clearly then, 
there are an infinite number of possible mirrors and the question then is which one to choose for a given 
application. This generally gives rise to the problem of finding a mirror shape given a prescribed property. 
To do this generally amounts to solving a differential equation. For example, if one requires a curve that 
focuses parallel rays, the answer is a parabola. In this case, the curve and it's special property are familiar 
and differential methods are not needed, but this is a rare exception. 

An early use of mirrors for panoramic imaging is a patent by Rees |12|, who proposes the use of a 
hyperbolic mirror for television viewing. Another patent is by Greguss 0, which is a system for panoramic 
viewing based on an annular lens combined with mirrored surfaces. 

Early applications to robotics using a conical mirror were carried by Yagi et al. in IT3l . Yamazawa et al 
[IT4l use a hyperbolic mirror for obstacle detection. 

In iflOl . Nayar describes an omnidirectional sensor from which it is possible to create perspective views 
by unwarping the images in software. In order to do this, the sensor must satisfy the single viewpoint 
constraint, which means that the sensor only measures the intensity of light passing through some fixed 
single point in space. This sensor uses a parabolic mirror, which, remarkably, is the only shape from which 
one can achieve a perspective unwarping of the image when using an orthographic camera (see (D). 

The first use of differential methods for design was by Chahl and Srinivasan in |2|. Here the authors 
employ differential methods to derive a mirror shape such that the radial angle and the angle of elevation 
(() (see figure {!}) are linearly related. 



In full . Ollis et al. argue that improved resolution uniformity is obtained if one demands that the angle of 
elevation § be proportional to the tangent of the radial angle 0; geometrically, tan(0) represents the distance 
of the image point to the optical axis. Thus, as the angle "sweeps" out some range of angle, a proportional 
length is swept out in the image plane (note that the line of reasoning here is essentially one dimensional). 
The authors then apply these mirrors to mobile robot navigation and panoramic stereo. 

A variant of the Ollis mirror is described by Conroy and Moore in Q. Using the same parameters 
as above, the authors then calculate what the mirror shape such that § is proportional to the area of the 
corresponding disk in the image plane. 

Hicks and Bajcsy derive the mirror shapes for a sensor in |7| which will give wide-angle perspective 
images, and so uniformly image planes. In 1 9 1 Hicks and Perline describe a sensor for which the projection 
map from the view sphere to the image plane is area preserving. This means that any two solid angles are 
allocated the same number of pixels by the sensor. 




Figure 1 : Chahl and Srinivasan consider a mirror in which i 



is a constant K. 
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Sensors designed to image cylinders were investigated by Gaechter and Pajdla in |5|. The analysis of 
these devices is essentially two dimensional and extended to three dimensions by radial symmetry. In fact, 
the unwarping map (corresponding to a cylindrical projection) for these sensors is the standard polar map. 

2.2 The Asymmetric Case 

If one leaves the realm of rotational symmetry, the problem of constructing a mirror for a given projection 
gets significantly more difficult. Not only are the equations much more complex, but with probability 1 they 
will not have solutions. 

In (HI this problem is addressed and methods for approximating solutions are introduced. Here we extend 
this work and give a new application. 

3 Problem Statement 

We work under the assumption that all cameras realize perfect perspective or orthographic projections. 

Suppose one is given a fixed surface S in R 3 and a camera with image plane /, also fixed in R 3 . A given 
mirrored surface M induces a transformation T M from some subset of / to S by following a ray from a point 
q E / until it intersects the mirror at a point r. The ray is then reflected according to the usual law that the 
angle of incidence is equal the angle of reflection and intersects S at a point s. We then define T M (q) = s . 



Figure 2: The statement of the most general form of the problem: given a transformation G from an image 
plane / to a surface S, find a mirror such that the induced optical transformation is as close to the prescribed 
transformation G as possible. 

The general problem is: 

Given G : / — > S, find M such that T M = G. If no such M exists, then find M such that T M is a good 
approximation to G. 

Figure © is a corresponding diagram. If an exact solution exists, then there are several ways to calculate it. 
Otherwise, there are numerous ways to formulate and solve the approximation problem. Below we briefly 
discuss three of these approaches - the fixed surface method, the vector field method, and the method of 



s 



s 
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distributions . If the problem does not have a solution, then each formulation gives rise to many possible 
choices of approximation techniques. 



3.1 The Fixed-Surface Method 

In this method, one assumes that a generic M is given in some coordinates and then from it calculate Tm. If 
one represents M as a graph z = f(x,y) then the expression for T M will contain partial derivatives of /. Thus 
the problem of finding M such that T M = G is reduced to the problem of simultaneously solving a system of 
partial differential equations. This is the approach introduced in 1 8 1 . 



3.2 The Vector-field Method 

Notice that for a given M, with q,r and s as above, that the vector + is normal to M at r. This 

suggest a method of constructing a vector field W on R 3 that will be normal to the solution: for each r e R 3 
define 

|q(r)-r| |G(q(r))-r| 

where q is the point in the image plane / corresponding to r (i.e. rays traced out from q contain r). If M 
exists and is expressed as a level surface F(x,y,z) = 0, then there will be a scalar function X(x,y,z) s.t. 



VF = XW. (2) 

In fact, several necessary and sufficient condition exist for testing for the existence of M from W. For 
example, M exists iff (V x W) • W = 0. 



3.3 The Method of Distributions 

In the above vector field formulation clearly W as defined is only parallel to VF and not generally equal to 
it. Thus the underlying structure of importance is the planar distribution <D that is orthogonal to W. Hence 
another formulation of the problem is to find an integral surface of the distribution <D (which is, of course, 
defined in terms of W). This may be then phrased as a variational problem, which may be attacked directly 
with numerical methods or one may consider the corresponding Euler-Lagrange equations. We will not 
discuss this method any further or give any examples here. 



4 Analog Panoramic Imaging 

The projection we are interested in mimicking with a mirror is the so-called cylindrical projection, in which 
the world is first projected onto a cylinder, which is the cut and rolled open, giving a panoramic "strip" 
(which for our purposes will lie in /). The resulting strip can be thought of as having coordinates r and 
where the coordinate varies from to 2% or some equivalent interval. Of course we cannot realize this 
projection exactly, since it has a single projection point, but we can approximate it very well. 

We take as our object surface S a cylinder whose axis of symmetry coincides with the optical axis of our 
camera, which we assume realizes an orthographic projection. This will allow us to apply the vector field 
approach. Despite the fact that we will use the vector field W described above, and not attempt to find an 
optimal X, the result is visually acceptable. 
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In figure Q appears three views of a mirror that has this property. This mirror provides a ±30 degree 
view in the vertical and a full 360 degree view in the horizontal. In figure © we see this surface used to 
image a chess board. To achieve this view the mirror is placed in the middle of the board (at approximately 
the height of the king's head) and viewed from below. 




Figure 3: Three different views of a surface which when used as a mirror gives a panoramic view without 
any digital unwarping. 




Figure 4: A panoramic view of a chessboard using the above mirror. 

Possible advantages of using a panoramic mirror of the above type over existing systems are that this 
sensor would not require a digital computer for unwarping, and that by performing the unwarping prior to 
the sampling of the image, the resolution of the image is more uniform. 

To find the above surface, we take the image plane, /, to be of the form x = c, so the optical axis is the 
x-axis. The surface S is taken to be a cylinder of radius p about the x-axis. The vector field W is derived 
as follows. According to our above formulation, r = [x,y,z] ,q = [c,y,z], and s = [pz,psin(y),pcos(y)]. (z 
plays the role of r and y the role of 6). At r the incident ray of light should be in the direction V where 



V= [pz,psin(y),pcos(y)] - [x,y,z] • 
(See figure ®.) Normalizing V and letting p — ► <x> gives the unit vector 

z sin(y) cos(y) 



U 



Then W = U + ei, i.e., 



W 



71+7' TIT? 

sin(y) cos(y) 



(3) 



(4) 



(5) 



.Vi+7 

Since we are free to scale W as we see fit and V(x — f{y,z)) = [— l,f yi f z ], a natural choice is to scale by 
the inverse of the first component of W, in which case the resulting vector field is 



i, 



sin(j) cos(y) 
z+\/l+z 2 'z+\/l+z 2 



(6) 
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( pz,psin(y),pcos(y^ff 




x = f(y,z) 



Figure 5: 



Thus the problem is to find a function f(y,z) whose gradient is as close to ^ 
Clearly there are different notions of closeness, but a very natural solution is to take g 



-sin(y) 



-cos(_y) 



as possible. 



h 



-'cos(j) 
z+VT+F 



-sin(y) 



and then take / to be the minimizer of 



/ (fy-g) 2 +(f Z -h) 2 dA. 
J A 



and 



(7) 



where A is a region in the y — z plane corresponding to the portion of th image plane / that is of interest (the 
panoramic strip) 1 . Presumably though we wish the y width of A to be at least 2n in order to get a panoramic 
image. The z component will control to vertical field of view. The above integral was minimized in Maple 
by taking / was to be a generic polynomial in y and z of fixed degree. Notice that the surface used in the 
ray tracing examples above was derived taking p — ► oo, and yet it works well for "close" objects - the mirror 
itself was approximately the size of a chess piece. 

Finally, the above sensor can be altered to fill an entire 640 by 480 video image entirely. In figure 
© we see several images of a mirror constructed by taking the union of the section of the above mirror 
corresponding to — | < y < | with its reflection, resulting a shape reminiscent of a conquistador helmet. 
This means that all the pixels of the video camera are being used to create the panoramic image. Compared 
with rotationally symmetric systems these lead to a 70 % increase in image resolution. Figure © shows the 
chess board using this mirror. 
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Figure 7: The chessboard viewed with the conquistador mirror. 
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